10/730,113 



MS306029.01/MSFTP2220US 



Amendments to the Claims 
This listing of claims will replace all prior versions of claims in the application: 
Listing of Claims: 

1 . (Currently amended) A method for evaluating a computer learning signal processing engine, 
comprising: 

identifying a first group of signal sets, each signal set of the first group having an 
associated range of values for a variable corresponding to the first group, the variable being one 
of a plurality of variables having values characterizing multiple signals to be processed; 

calculating an accuracy score for each signal set of the first group using the signal 
processing engine to be evaluated; 

applying weight factors to the accuracy scores for the first group signal sets, each weight 
factor representing a relative importance of one of the associated ranges of values for the first 
variable; 

summing weighted accuracy scores for the first group of signal sets to yield a first 
summed accuracy score; 

identifying additional groups of signal sets, each group having a corresponding variable of 
the plurality of variables, each signal set of a group having an associated range of values for the 
corresponding variable; 

calculating accuracy scores for each signal set of each additional group using the signal 
processing engine to be evaluated; 

applying weight factors to the accuracy scores for the signal sets of the additional groups; 

summing the weighted accuracy scores within each of the additional groups to yield 
additional summed accuracy scores; 

and using summed accuracy scores from at least two separate training sets comprising 
samples, wherein each separate training set is distinguished by a feature characteristic identified 
based upon a demographic characteristic associated with a source of the samples, to create one or 
more signal processing engines to handle multiple applications to one or more new groups of 
signal sets for which a frequency of features the feature characteristic of the separate training sets 
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are known or assumed, by weighting the summed accuracy score associated with each training 
set according to the frequency and then combining the weighted summed accuracy scores. 

2. (Original) The method of claim 1 , wherein at least some of the associated ranges of values 
are single values. 

3 . (Original) The method of claim 1 , wherein said further combining the summed accuracy 
scores comprises: 

applying weight factors to a plurality of the summed accuracy scores, each of the 
weight factors applied to the plurality of summed accuracy scores representing a relative importance 
of the variable corresponding to the group from which a different one of the plurality of summed 
accuracy scores was derived, and 

summing the plurality of weighted summed accuracy scores to yield a 
subsequent sum. 

4. (Previously Presented) The method of claim 3, wherein said method further combining the 
summed accuracy scores comprises: 

applying additional weight factors to the subsequent sum and to at least one of the 
summed accuracy scores, the additional weight factors representing the relative importance of at 
least one variable corresponding to groups from which the subsequent sum was derived and of the 
variable corresponding to the group from which the at least one summed accuracy score was 
derived, and 

summing the additionally weighted subsequent sum and the additionally 
weighted at least one summed accuracy score. 

5. (Original) The method of claim 3, further comprising: 

identifying an additional variable having values characterizing multiple signals to be 
processed; 

identifying another group of signal sets, each signal set of the group having an 
associated range of values for the additional variable; 

calculating an accuracy score for each signal set of the additional variable group using the 
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signal processing engine to be evaluated; 

applying weight factors to the accuracy scores for the additional group signal sets, each 
weight factor representing a relative importance of one of the associated ranges of values for the 
additional variable; 

summing weighted accuracy scores for additional group signal sets to yield an 
additional summed accuracy score; 

applying weight factors to the plurality of the summed accuracy scores and to the 
additional summed accuracy score, each of the weight factors representing a relative importance 
of the variable corresponding to the group from which a different one of the plurality of summed 
accuracy scores was derived or of the additional variable, and 

summing the plurality of weighted summed accuracy scores and the weighted 
additional accuracy score. 

6. (Original) The method of claim 1 , further comprising 
selecting a variable of the plurality of variables; 

selecting sub-variables, each sub-variable having a range of values for a value of the 
selected variable; 

calculating an accuracy score for signal sets in groups of signal sets corresponding to the 
sub-variables, each of the signal sets within a group corresponding to a range of values for the 
corresponding sub-variable; 

applying weights to the accuracy scores for the signal sets in each sub-variable group and 
summing the weighted scores within each of said groups to yield sub-variable accuracy scores; 
and 

applying weights to the sub-variable accuracy scores and summing the weighted sub-variable 
accuracy scores. 

7. (Original) The method of claim 1 , wherein: 

at least one variable is a source variable having values characterizing a source of a 
signal to be processed, 

at least one variable is a context variable having values characterizing the 
context of a signal to be processed, and 
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at least one variable is a physical variable having values characterizing 
physical attributes of a signal to be processed. 

8. (Original) The method of claim 7, wherein at least one variable has values characterizing a 
user scenario in which a signal was generated, the user scenario values including at least one of a 
software application and an operation performed within a software application. 

9. (Original) The method of claim 8, wherein: 

the signal processing engine to be evaluated comprises handwriting 
recognition software, 

the signals to be processed comprise handwriting samples, 

values of the source variable comprise demographic data regarding users 
creating handwriting samples, 

values of the context variable comprise data regarding the context of 
handwriting samples, and 

values of the physical variable comprise at least one of data regarding the scaling of 
a handwriting sample, data regarding the relative angle of components of a handwriting sample 
and data regarding the spacing between components of a handwriting sample. 

10. (Original) The method of claim 9, wherein: 
the groups of signal sets comprise: 

groups of demographic signal sets, each demographic signal set in a group having an 
associated range of values for the source variable and the same value for the context variable, 

groups of scaling signal sets, each scaling signal set in a group having an associated range 
of values for the scaling of a handwriting sample and the same value for the context variable, and 

groups of angle signal sets, each angle signal set in a group having an associated range of 
values for at least one angle of a component of a handwriting sample and the same value for the 
context variable, 

at least the summed accuracy scores for demographic signal set, scaling signal set 
and angle signal set groups of signals having a common context variable value are weighted and 
summed to yield context accuracy scores, 
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context accuracy scores are weighted and summed to yield a combined context 
accuracy score, 

the combined context accuracy score and at least one other summed accuracy score are 
weighted and summed to yield a word accuracy score, and 

at least the word accuracy score and a summed accuracy score for a group 
corresponding to the user scenario variable are weighted and summed to yield an overall 
accuracy score. 

1 1 . (Original) The method of claim 8, wherein: 

the signal processing engine to be evaluated comprises speech recognition 
software, 

the signals to be processed comprise speech samples, 

values of the source variable comprise demographic data regarding users creating 
speech samples, 

values of the context variable comprise data regarding the context of speech 
samples, and 

values of the physical variable comprise at least one of data regarding the scaling of a 
speech sample and data regarding the spacing between components of a speech sample. 

12. (Original) The method of claim 11, wherein: 
the groups of signal sets comprise: 

groups of demographic signal sets, each demographic signal set in a group having an 
associated range of values for the source variable and the same value for the context variable, and 

groups of scaling signal sets, each scaling signal set in a group having an associated range 
of values for the scaling of a speech sample and the same value for the context variable, 

at least the summed accuracy scores for demographic signal set and scaling signal set 
groups of signals having a common context variable value are weighted and summed to yield 
context accuracy scores, 

context accuracy scores are weighted and summed to yield a combined context accuracy 

score, 

the combined context accuracy score and at least one other summed accuracy score are 
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weighted and summed to yield a word accuracy score, and 

at least the word accuracy score and a summed accuracy score for a group corresponding to 
the user scenario variable are weighted and summed to yield an overall accuracy score. 

1 3 . (Original) The method of claim 1 , further comprising applying a transforming function to a 
sum. 

14. (Original) The method of claim 13, wherein the transform function comprises: 
outputting the sum if the sum is above a threshold, and 

outputting number having a large absolute value if the sum is not above the 
threshold. 

15. (Original) The method of claim 1 , further comprising: setting accuracy scores for signal sets 
to equal 1; applying confidence scores to weight factors; and calculating an overall confidence 
score. 

16. (Currently amended) A computer-readable medium having stored thereon data representing 
sequences of instructions which, when executed by a processor, cause the processor to perform 
steps comprising: 

identifying a first group of signal sets, each signal set of the first group having an 
associated range of values for a variable corresponding to the first group, the variable being one of 
a plurality of variables having values characterizing multiple signals to be processed; 

calculating an accuracy score for each signal set of the first group using a signal 
processing engine to be evaluated; 

applying weight factors to the accuracy scores for the first group signal sets, each 
weight factor representing a relative importance of one of the associated ranges of values for the 
first variable; 

slimming summing weighted accuracy scores for first group signal sets to yield a first 
summed accuracy score; 

identifying additional groups of signal sets, each group having a corresponding 
variable of the plurality, each signal set of a group having an associated range of values for the 
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corresponding variable; 

calculating accuracy scores for each signal set of each additional group using the 
signal processing engine to be evaluated; 

applying weight factors to the accuracy scores for the signal sets of the additional 
groups, the weight factors within each of the additional groups representing a relative importance 
of associated ranges of values for the variable corresponding to the group; 

summing the weighted accuracy scores within each of the additional groups to yield 
additional summed accuracy scores; and 

and using summed accuracy scores from at least two separate training sets comprising 
samples, each separate training set is identified by a feature characteristic determined based upon 
a demographic characteristic associated with a source of the samples, to create one or more 
signal processing engines to handle multiple applications to one or more new groups of signal 
sets for which a frequency of features the feature characteristic of the separate training sets are 
known or assumed, by weighting the summed accuracy score associated with each training set 
according to the frequency and then combining the weighted summed accuracy scores, furth e r 
combining the summed accuracy scores. 

17. (Original) The computer-readable medium of claim 16, wherein at least some of the 
associated ranges of values are single values. 

18. (Original) The computer-readable medium of claim 16, wherein said further combining 
the summed accuracy scores comprises: 

applying weight factors to a plurality of the summed accuracy scores, each of the weight factors 
applied to the plurality of summed accuracy scores representing a relative importance of the 
variable corresponding to the group from which a different one of the plurality of summed 
accuracy scores was derived, and 

summing the plurality of weighted summed accuracy scores to yield a 
subsequent sum. 

19. (Original) The computer-readable medium of claim 18, wherein said further combining 
the summed accuracy scores comprises: 
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applying, additional weight factors to the subsequent sum and to at least one of the 
summed accuracy scores, the additional weight factors representing the relative importance of at 
least one variable corresponding to groups from which the subsequent sum was derived and of 
the variable corresponding to the group from which the at least one summed accuracy score 
was derived, and 

summing the additionally weighted subsequent sum and the additionally 
weighted at least one summed accuracy score. 

20. (Original) The computer-readable medium of claim 18, comprising additional data 
representing sequences of instructions which, when executed by a processor, cause the 
processor to perform additional steps comprising: 

identifying an additional variable having values characterizing multiple signals to be 
processed; 

identifying another group of signal sets, each signal set of the group having an 
associated range of values for the additional variable; 

calculating an accuracy score for each signal set of the additional variable group using the 
signal processing engine to be evaluated; 

applying weight factors to the accuracy scores for the additional group signal sets, each 
weight factor representing a relative importance of one of the associated ranges of values for the 
additional variable; 

summing weighted accuracy scores for additional group signal sets to yield an 
additional summed accuracy score; 

applying weight factors to the plurality of the summed accuracy scores and to the 
additional summed accuracy score, each of the weight factors representing a relative importance 
of the variable corresponding to the group from which a different one of the plurality of summed 
accuracy scores was derived or of the additional variable, and 

summing the plurality of weighted summed accuracy scores and the weighted 
additional accuracy score. 

21 . (Original) The computer-readable medium of claim 16, comprising additional data 
representing sequences of instructions which, when executed by a processor, cause the processor 
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to perform additional steps comprising: 
selecting a variable of the plurality; 

selecting sub-variables, each sub-variable having a range of values for a value of the 
selected variable; 

calculating an accuracy score for signal sets in groups of signal sets corresponding to the 
sub-variables, each of the signal sets within a group corresponding to a range of values for the 
corresponding sub-variable; 

applying weights to the accuracy scores for the signal sets in each sub-variable group and 
summing the weighted scores within each of said groups to yield sub-variable accuracy scores; 
and 

applying weights to the sub-variable accuracy scores and summing the weighted sub-variable 
accuracy scores. 

22. (Original) The computer-readable medium of claim 16, wherein: 

at least one variable is a source variable having values characterizing a source of a 
signal to be processed, 

at least one variable is a context variable having values characterizing the context 
of a signal to be processed, and 

at least one variable is a physical variable having values characterizing 
physical attributes of a signal to be processed. 

23 . (Original) The computer-readable medium of claim 22, wherein at least one variable has 
values characterizing a user scenario in which a signal was generated, the user scenario values 
including at least one of a software application and an operation performed within a software 
application. 

24. (Original) The computer-readable medium of claim 23, wherein: 

the signal processing engine to be evaluated comprises handwriting 
recognition software, 

the signals to be processed comprise handwriting samples, 

values of the source variable comprise demographic data regarding users 
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creating handwriting samples, 

values of the context variable comprise data regarding the context of 
handwriting samples, and 

values of the physical variable comprise at least one of data regarding the scaling of 
a handwriting sample, data regarding the relative angle of components of a handwriting sample 
and data regarding the spacing between components of a handwriting sample. 

25. (Original) The computer-readable medium of claim 24, wherein: 
the groups of signal sets comprise: 

groups of demographic signal sets, each demographic signal set in a group having an 
associated range of values for the source variable and the same value for the context variable, 

groups of scaling signal sets, each scaling signal set in a group having an associated range 
of values for the scaling of a handwriting sample and the same value for the context variable, and 

groups of angle signal sets, each angle signal set in a group having an associated range of 
values for at least one angle of a component of a handwriting sample and the same value for the 
context variable, 

at least the summed accuracy scores for demographic signal set, scaling signal set and 
angle signal set groups of signals having a common context variable value are weighted and 
summed to yield context accuracy scores, 

context accuracy scores are weighted and summed to yield a combined context accuracy 

score, 

the combined context accuracy score and at least one other summed accuracy score are 
weighted and summed to yield a word accuracy score, and 

at least the word accuracy score and a summed accuracy score for a group corresponding to 
the user scenario variable are weighted and summed to yield an overall accuracy score. 

26. (Original) The computer-readable medium of claim 23, wherein: 

the signal processing engine to be evaluated comprises speech recognition 
software, 

the signals to be processed comprise speech samples, 

values of the source variable comprise demographic data regarding users creating 
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speech samples, 

values of the context variable comprise data regarding the context of speech 
samples, and 

values of the physical variable comprise at least one of data regarding the scaling of a 
speech sample and data regarding the spacing between components of a speech sample. 

27. (Original) The computer-readable medium of claim 26, wherein: 
the groups of signal sets comprise: 

groups of demographic signal sets, each demographic signal set in a group having an 
associated range of values for the source variable and the same value for the context variable, and 

groups of scaling signal sets, each scaling signal set in a group having an associated range 
of values for the scaling of a speech sample and the same value for the context variable, 

at least the summed accuracy scores for demographic signal set and scaling signal set 
groups of signals having a common context variable value are weighted and summed to yield 
context accuracy scores, 

context accuracy scores are weighted and summed to yield a combined context accuracy 

score, 

the combined context accuracy score and at least one other summed accuracy score are 
weighted and summed to yield a word accuracy score, and 

at least the word accuracy score and a summed accuracy score for a group corresponding to 
the user scenario variable are weighted and summed to yield an overall accuracy score. 

28. (Original) The computer-readable medium of claim 16, comprising additional data 
representing sequences of instructions which, when executed by a processor, cause the processor 
to perform additional steps comprising applying a transforming function to a sum. 
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29. (Original) The computer-readable medium of claim 28, wherein the transform 
function comprises: 

outputting the sum if the sum is above a threshold, and 

outputting number having a large absolute value if the sum is not above the 

threshold. 

30. (Original) The computer-readable medium of claim 16, comprising additional data 
representing 

sequences of instructions which, when executed by a processor, cause the processor to 
perform additional steps comprising: 

setting accuracy scores for signal sets to equal 1 ; 
applying confidence scores to weight factors; and 
calculating an overall confidence score. 
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